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ABSTRACT 

One purpose of this project was to develop positive 
attitudes and knowledge of evaluation philosophy and techniques at 
the local building level. A set of procedures and instruments was 
devised through the efforts of the project staff to train 
participants to plan and carry out activities appropriate for 
assessing the particular educational needs in their school. 
Evaluation activities were implemented by a building committee formed 
in each elementary school in conjunction with project staff. A 
reading assessment instrument was provided as a basis for the 
committee to plan the evaluation. The six sections contained in the 
instrument were designed to aid the collection of information 
concerning student performance, teacher and administrator strengths 
and weaknesses, adequacy of instruction materials, and parental 
influences in the reading program. The results of the first year 
procedure indicated that the section of the reading instrument 
concerning teacher strengths and weaknesses was useful to 
administrations in planning for staff development programs. Also, in 
schools where comprehensive evaluations based on the suggestions from 
the student strength and weakness section were planned, a great 
amount of time and thinking were required to collect data relevant to 
their question. (HR) 
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SECTION I - Specifications of the Project 



A. Statement of Purooae 

Pjroject PRIMES provided facilitating «nd direct aervlc^a to the public 
•nd parochial cchoola in Columbui attempting to fulfill mandated evaluation 
requirements for meeting Ohio Standards for filementary Schools* 

One purpose of the project was to develop positive attitudes and knowledge 
of evaluation philosophy and techniques at the local building level. A set of 
procedures and Instruments were devised through the efforts of the project staff 
to train participants to plan and carry out activities appropriate foV assessing 
the particular educational needs in their school, 

B« Procedures 

Evaluation activities were implemented by a building coiwnittee formed in 
each elementary school In conjunction with project staff. During the yeari 
1972-73, the curriculum component of school operation was the state mandated 
area chosen for assessment. From the various content fields the school conmittee 
composed of jprlnclpal, teachers and lay persons selected a focus for evaluation. 
Reading, the most frequent choice, was studied by 87 of the 151 participating 
. schools. 

The procedures planned for project implementation determined to some extent 
the amount of staff assistance to individual schools. A single briefing for the 
purpose of designating procedures and reviewing the Instrument was required of 
principals who chose to carry out evaluation activities on their own. A second 
approach was to provide evaluation planning assistance to the building conrolttee 
following the session with the principal. Finally, a procedure which assurt/d 
the assistance of the project staff throughout the total schedule of evaluation 
activities was possible. Of thi 87 schools selecting to study reading, 46 
principals decided to direct the evaluation activities themselves, 19 asked for 
help in planning with their committees and staff, and 16 requested assistance 
through the total evaluation process. Reading evaluation requirements were 
waived in 2 schools* where pilot programs requiring evaluation were not under 
way. At the end of the year four echools had failed to fulfill project requirements. 

During the planning meeting the committee reviewed the instrument and 
decided upon the sections they would use. The collection and tabulation of 
responses to the various sections was co-ordinated and reported according to 
coiflmittee" specifications. 

Tnstr umen ta tipn 

"A reading assessment Instrument was provided as .a ba^ls for ih«^ commiitee 
to plori'jthe ovftlualJoh. Iho ifetlonf contMn«;(l \t\ Iho {hfitnm-fil woro 




tans. fabte^f'^deplBtsUfie 'number of lljhodU' Using each 
" the lhst)rufflint fpi* \ey4Vu|iioh i^u 



Table I 



Building Evaluation Connittees SeUetion of 
Sections of Reading Assessment Instrument 



Typa of Information 



Number of Schools* 



Pupil Performance and Attitudes 

Teacher Strengths and Weaknesses 

A<faninl8trator Strengths and Weaknesses 

Adequacy of Materials 

Parent Questionnaire 

Factors Preventing Optiimim Program 



39 

f$ 
64 
67 
68 
35 



*The total number of schools completing evaluations of the reading program was 81. 



A* Svaluation Activitie s of Participants 

The Information gathered was intended to serve the purpose of Individual 
schools in planning for educational Iroprovementi thus data particular to a 
given school was interpreted and presented in the school's evaluation reporti 
Results from participating schools were compiled for this report and in general 
present a positive view toward reading programs accross the school system. 

Information is presented in relation to the various sections of the 
reading assessment instrunent. 

1* Pupil Strengths and Weaknesses 

Section A of the assessment instrument listed several suggestion)} 
for gathering information about the level of student performance in 
reading. Building committees were not required to use this section in 
the evaluation designi If they did choose, one or more of the items 
could be selected as a basis for student assessment. There were 39 
schools in which one or another of the suggestions .were discussed and 
determined. Specific suggestions and the number of schools which ' 
reported tWe collection of such data are shown in Table 2, 

2. Teacher Strengths and Weaknesses 

Section B of the reading assesament instrument was most frequently 
used In reading prograiij asses^i^nV^ a 



SECTION II - Results 
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Tabid 2 

Typ«8 ot Student Information Reportedly 
Used to Assesfl Pupil Strengths and Weakr.es8e8 
In Columbus Elementery Schools. 



Information Used for Assessing Students 


^^Mlber of Schoole 


1. 


Comparison of grade level achievement 
scores with grade level potential (baseci on 
mental maturity testing) 


29 




iaenxiTica^ion oti numper of under- achieving 
students in each classroom 


32 


3. 


Identification of specific areas of reading 
airTicuix;y ve*g» meaning vocabulary as shown 
by standardized test printout) 


28 


4. 


Identification of specific weaknesses for 
lndividua.V children as shown by standardized 
test printout 


28 


5. 


Use of item-analysis provided for grades 4*6 to 
show difficulties in vocabulary or comprehen- 
sion. 




6. 


Use reading readiness test in assessment of 
kindergarten program 


20 


7. 


Consider pupil ettltudes toward reading 






Table 3 



Frequency of Negative and Poeitlve Reeponsei 
By School* From Teacher Self- Eva luatlcn 



Item 


Number of Schools 






No 


!• Genuine Interest in teaching reading 


70 


0 


2. Training! 

a) general course in elementary 












teaching 


ov 


0 


"! 


language arts methods course 


oV 


0 




languea^ arts and reading methods 


Of 


:i 


d 


pbychpiogy ef reading 




'111 




a reading methods course 




\ 


. f; 


children literature 


uo 


1 


i 


corrective reading 


18 


01 




in-service course on reading 






3. Experience 1 






a. 


class as a whole In basal r^eader 


60 


V 


b; 


^ reading groups in basal rel^der ^ 


69 


A 

u 


c. 


individualized reading program 


56 


13 


d; 


programmed reading 




49 


e. 






PO 


f| 


Linguistics 


13 


Do 




team teaching 


111 

iO 




h 


co«*operative teaching 


wQ 


33 


1 


departmentalized teaching 


IQ 


d3 




Joplin Plan 


Q 


01 


k 


teaching machines 




57 


i] 


» tutoring 




AZ 

26 


4. Knowledge and understanding of total 






reading program 






a 


\ goals 


64 


5 


b; 


se<)uence of skill development 


65 


4 


c] 


1 necessity of teaching 








individual students 


67 


2 


d\ 


1 methods 


66 


3 


e, 


1 organizational patterns 


' 64 


5 


f! 


1 evaluation 


61 




5* Rating 'Strengths and weaknesses In 


Strong 


Weak 


teaching reading skills 












1 pre*readlng 


46 






f concepts -meaning vocabulary 


65 


4 


c] 


► sight vocabtiUry 


67 






66 




^1 




64 


5 



Tabu 3 (Cont'd) 



6* Ablt to diagnose strengths and weaknesses 

In individual students 
7» Abl« to pr&scribe instruction to meet 

Individual student needs 
8. Able to evaluate progran effectiveness 
11« Abl* to co-ordinate existing reading 
services effectively 
a) pre-kindergarten 

d) reading resource teacher 

e) reading clinic 

f) speech therapist 

g) educational aides 
hV volunteers 

i) tutors 
16t Read aloud to class daily 



Number of Schools 
Strong 



67 

64 
67 



21 
49 
36 
63 
50 
41 
54 
65 



Reporting 
Weak 



A 



5 

2 



48 
20 
33 

6 
19 
28 
15 

4 



* Diocesan schools are not included in Table presentation 



At 76 schools teachers responded to C|uest ions concerning their training 
and experiences in teaching reading* Self-evaluative responses about knowledge 
of various program components and instructional expertise were collected, also. 
For most classroom teachers* university preparation included 1) a general course 
in ele«entary teaching, 2) a language arts method course* 3) a language arts and 
reading methods course, 4) a reading method course and 5) children's literature < 
Cour'ses not included in their training were psychology of reading and corrective 
readihlir 



Bxp«rieneoi in ttachlng reading reborted by classroom teachara wsta 
I) elaaa as a whola in basal raadarst 2) reading groups in basal readersi 3) 
indf.viduali2ed reading programs^ 4) co-operative teaching and 9) tutoring* 

The majority of respondents said they had had no experience with progranMed 
readingi XtA| UnguisticSf team teachingi departmentalized teachings the Joplin 
Plan^ and teaching machines* 

Teacher responses Indicated confidence in their knowledge and understanding 
of the total program and in their ability to instruct and evaluate in reading. 
Satisfaction wss indicated with the results of referr/ils made to supportive ser* 
vices for pupils with problems* Efforts to co-ordinate the services of reading 
teachersi reading resource teachers^ speech therapistt^f educational aidesi 
volunteers and tutors were said to be effective* Difficulty was noted In ef forte 
to use effectively the services of pre*school programs and reading clinics* 

Based on the high number of positive responses to items 1 and; 14 elementary 
reading programs in the schools represented would seem to expose puplU to good 
adult attitudes and interest in readings* 

In all the schools teachers reported that they communicated with parents 
by note and telephone in addition to progress reports and conferences* Some 
teachers in all but nine schools had made home visits* 

3 * Administrator Strengths and Weaknesses 

Implementation of the reading instrument # Section Of was 
accomplished by sixty building evaluation corrmittees* 

Forty-one of these schools asked the administrator to respond 
in a self«*eyaluative manner* The regaining 19 schools had teachers 
respond according to thtir impressiona of the ad»inistrator# Both 
approaches resulted in favorable views of the principals ability 
to provide leadership for the school reading program* Table 4 
depicts some of the important items and the frequency of positive 
and negative responses to them {Parochial school data not included 
because of difference in compiling)* 

Another item In the questionnaire was concerned with reasons why pupils 
did not receive adequate support services* 

ThreQ types of problems were reported 

1) unavailability of services because of inadequate space or 
facilities (waiting lists, full program^ etc.) 

2) complexity of referraJ procedures 

3) unco-operative parents (difficulty of accepting problems of 
child)* 



Table 4 
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Frequency of Betponeee by Principals or Staff to 
Items from Section C Administrator Strengths and 

Weaknesses 



Itewa 


Administrators 


staffs 


Yes 


No 


Yes 




1. 


Knowledge and understand ing of 












total reading progrant 






15 






a 


difficulties of teaching individually 


40 


0 


0 




b! 


stqiuence of skills 




A 

V/ 


14 






: c 


methods 


36 




14 


1 




dl 


materials 




6 


14 


1 




e! 


goals 


3g 


2 


1ft 


0 




f 


evaluation 


3d 


3 


12 


2 




g- 


organizational patterns 


35 




15 


0 




Leadership provided by 

a) encouraging experimentation . 














or innovation « ^ 


37 


4 


19 


0 






\ provision for material and use 


40 


1 


14 


1 




1 


\ in-service progr^^ns 


31 


10 


13 


2 






1 constructive suggestions 


37 


4 


13 


2 




e] 


1 flexible organizing for instruction 


40 


I 


15 


0 


3. 


Provido assistance for teachers having 












problems with reading instruction 


37 


4 


12 


3 


4. 


Coordinate existing services t 














1 reading teacher 


40 


.1 


19 


0 




•J: 


1 speech therapist 


39 


2 


15 


0 






\ psychologist 


40 


1 


14 


1 




d 


1 reading resource teachdrs 


37 


4 


14 


1 




e 


f educational aides* 


30 


4 


\A 


1 




: f 


) tutors 


36 


5 


14 


1 




9 


\ : library aides* 


27 


7 


10 


2 




h 


\ volunteers* 


32 


6 


13 


1 




i 


V Riaht to Read 


20 


n 


7 


3 



* Not applicable in some schools 
4. Instructional Materials 

Tlie section listing a variety of instructional materlaU« Section Pi 
was incorporated in the asses^flifent plan of 66 participating schools. Courses 
of study and curriculum guides as well as basic tests were said by most 
schools to be in adequate supply* However a large number of the schQpU 
reported that diagnostic instruments to aid in determining teaching I* 
approaches were unavailable* Diagnostic Instruments for deteciiii hi hg pupil 
strengths and weaknesses and materials for teaching baaic skills were 



aviilable to the schools. Although i f ew stho^U^^rfpillM^^ 
adeqOately Supplied with earphohei rtsentiSl to in|tructlon* for the 
Houghion Mifflin series-visual equipment and materials. 




Table 5 

Nt^ber of Schools Reporting Instructional 
Materials to be Adequate or Inadequate 



Types of Materials 



NuBiber of Schools 



li Course of studyt curriculum guide 
2t Basic Texts 

3* Diagnostic instruments to detersiine 
approach 

e) sight 

bV phonetic 

cj kinesthetic 

dV cottbination 
4# Diagnostic instruments to determine 

pupil strengths and weaknesses 
5# Materials for basic reading skills 

motor 

visual (pre-reading 
auditing \ 

concept S'^meaning vocabulary 
sight vocabulary 
comprehension 
study skills 
flexibility of rate 
6« Basic audio^visual equipment and mate^ 
. rials. : 

tape-' recorder 
earphones 
movie projector 
fllmstrip projector 
record player 
TV 

^» High Interest-^low read-ability books 

B. Paperback library 

^« Heading games and devices 

10» Puppets 

Ui Pictures and dictlonalres 
12* Reference Materials 
13* Books for Recreational Reading 
14» Programmed Learning Materials 
15. Workbooks-Worksheets 

It* iutilun fi6aW 
i8# Magnetic boards 



c) 
d) 
e) 
f) 



62 
65 



42 
41 

27 
31 

52 

39 
55 
52 

59 
60 
59 
56 
49 



58 
56 
64 
64 

62 
60 
SO 
24 
49 
28 
62 
61 
64 
34 
54 
54 
66 
42 
61 

ii 



TabU 6 



Frequency of School Response to ItMS on Pomt QuHtlonniiro 



Hm 



l» Do you xe«d to your child? 
2. Did child viow •ducnional TV 

(l*** S6i«mo Street)? 
3* Did you tiko child tot coMMihlty 

tri|>$? 

4. Did you telk about trip with him? 

5* Do you road together? 

6* Vielt library with child? 

7. Provide place for child to study? 

8* Does child read to you 

himteU 
broihere aftd 
eleteri 

9. Does teacher give you speoific 

suggestions for helping your child? 
10* Have you observed reading being 

taught in school? 
n. Are you satisfied with reading 

program? 
\2» Do you know what should b^ 

taught at each grade level? 
il3, Dg» you have opportunities to 
'V 5 express opinions ai^d make reeonnen- 
, dations about you* child's reading 

program? 

14* Are new reading programs explained? 



64 

64 

63 
61 
62 
46 
62 
64 
64 

60 

61 

24 

56 

12 



54 

37 



5. Parent Quest ionntare 

^ull^lrif 0v'|lUftlon CoiiMitteei in 64 schools sa»plfid parent opinion 
in assessing the scMl readlno progifarn* Twenty-one 6f the sthoMs ||l(ed 

cortftltteiryJlir^bU to^btlin VPi^t^N^^ the t#lc^l p^r,ehi*JnHhe 




lERIC^^Ii 



It is noted that th« abovd numbers fHt to schools reporting msjority 
responses* Thus ditA frOM psrtioular schools might 9lve soraewhtiit dlfferi^hi 
views than that presented In the chart* Project staff attempted to aid thiv 
evaluation coiwhltteo In Interpreting date f r<Mn particular schools whlchfliili 
might reflect perehtal dissatisfaction o» dlsintereit* In several scho^lllljiy 
which one or more of the Items got many negative responses* altiiough not'«*^% 
necessarily a majority, Attempts were made to Inform parents, through special 
prograrcs or literature, about those things which' they did not, know* 

6. Factors Preventing the p^velopment of an Optimal Reading Program* ' 

This was an open-ended question to which thirty five schools 
responded for assessment purposes* Table 7 displays the factors 
listed consistently by responding schools as Urge class size, 
the need for more professional ahd tutorial resource persons. 
Several schools noted the lack of libraries and listening centers* 
Physical space, Inadequate planning and instructing time and attitudes 
and knowledge of professiortdls were listed by over half of the respond-^ 
ing schools* 



Table 7 

Frequency of Factors Preventing Optlmsl Reading 
Programs 



Factor 


Number of Schooi'i Llstlrtg Factor 


Profdssional and/or tutorial resource 
persons 


28 


More and Better Materials and 
Bquipcnent 


27 


Large Class S)lxe 


24 


Libraries and Listening Centers 




Inadequate Planning and Instructing 
Tltne 


12 


Attitudes and Knowledge of 


11 
10 




■ 10 



SECTION III— Suffliflary and Recommendation 



A* Val»e of Out^on^f^ 

On the whol« the flrat year procedure and Inetrgment provided an 
appropriate degree of participation for the schools fulfil ling the evaluation 
n^CrSi®* ^® "^^"^ required and the level of skill needed for implementing 
PRIMES were acceptable to elementary principals and staffs. Quite frequently 
follow-up activities based on evaluation outcomes were meaningfully planned 
and carried out at the building level. The section of the reading instrument 
concerning teacher strengths and weaknesses was especially useful to administra- 
tors in planning for staff development programs. 

Evaluation data coUected with the reading instrument tended to present a 
uniform picture, with emphasis on teacher characteristics and instructional 
materials, of programs throughout the system. Markedly different approaches 
and organlMtlonal patterns and the particular problems of Individual schools 
were barely reflected In the assessment. In the few schools where comprehensive 
evaluations based on suggestions from the student strength and weakness section 
were planned, it was discovered that a grest amount of time and thinking were 
required to collect data relevant to their questions, and to plan for approprloti 
changes based on the results. 

B. Relevancy of Oblectlves 

Experi#ncei this year In. assisting and facilitating evliluatlon activities 
seem to demonstrate a need for Increased knowledge of and better training for 
evaluatipn on the part of many project pariiclpants. Assistance ssemed 
essential in many schools because of the r^fstriints of time imposed by dally 
administrative and Instructional resppnSlbiUtles. The Impltmentation of 
meaningful and comprehensive evaluation designs required careful planning and 
scheduling. For these reasons the goal of PRIMES to provide training and 
assistance to individual schools fulfilling the evaluation mandate Is. very 
appropriate. 



^ Project eotlvltlee and evaluation outcoisea euggest oeveral levels of 
education to ^hU\ - ecofniBendations night be addressedi 

lt» Project -*vel - A recommendation for the project is that the consulting 
services of the project staff be more efficiently distributed to schools 
!J .rS evaluation asjlstanco. The basis for consulting ass.ign,«ients 
should be considered in light of the amount of service revested by 
specific schools to accofliplish evaluation activities rather than on the 
subject area selected for assessment* The services of one project staff 
member were directed toward the majority of the participating schools 
an choosing to focus on reading program evaluation during the Initial 
project year. It was difficult to provide the amount of assistance 
needed and/or requested to realize their goals for assessment* 

lb. A second recommendation directed to the project is concerned with 
the development of an evaluation instrument or a set of procedures 
which will be flexible enough to initiate evaluation activities In 
very diversified educational communities, but which will provide useful 
information to specific schools. Staff objectives for the Initial 
project year demanded a great deal of effort in revising pilot Instruments. 
It is reconnended that these efforte be continued with syste-' 
matic evaluation of thd implementation of revised instruments durino 
year two. ' 

2. Building Level - At the building level the cohtinuatiort of project aotivi- 
ties in order to accomplish state requirementi for evaluation is strongly 
recommended! As defined operat^oniUy by state standards, systematic 
evaluation is a 8ome*het unfamiliar process to elementary sch6<lr stiffs. 
The number of schooh unwilling or unable to collect and use in^oniiation 
concerning student achievement and sptltude is evidence of a need W 
further training and practice in the use and Interpretation of assessment 
techniques. Assisting and facilitating services to increase t^chnicii an 
interpretive skills as well as to stimulate motivation in eieme'"^'-"'**' ' 
staffs is important. 



3. System Level - At the system level» it is reconmended that support and 
encouragement be given to the efforts of individual schools to Interpret 
and use utudent data for improving inatruction. Section 6, b, and P of <he 
reading Instrument provided information concerning staff training and 
in-service. These sections pointed out in different ways the need for 
more training in specific areas of reading instruction. Difficulties 
facing principals and teachers are planning, grouping and orgafii zing for 
more accurately prescribed reading instruction for IndivlduM^studente. as 
well as, for particular schools. AltHoOgh instructional materials Wire 
fenera I ly considered to be in adequate supply, more ia^ficieHt and i^lfective 
use of materials was a matter of concern. In-service progHms mlgftt well 
be planned around the fe concerns. 



4. State' UQT v education in Ohi^'^KWmlndated 

^^ilMiSfiSllflliiy^^* Niy^nd tfe c>pabif?tfil Sf^'Jf^^ 
?*lffl3pffii|^P® time forevllWoiS are Int^lved.'wt^ 
recomm*r|fra|^#IU depa^tiiieftt of |au4ttWffo#S-»# 

.el|anef.filVlte|T^ systems in deviio|inrihT«els|¥^ if^ 

^ill^|ilij!fl^ttl^rg the Itaftdards. 
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